perm filename PROPOS[8,ALS] blob sn#043237 filedate 1973-05-17 generic text, type T, neo UTF8
00010									May 16 1973
00020	
00030	 A Proposal for Speech Understanding Research
00040	
00050	
00060		It is proposed that the work on speech recognition that is
00070	now under way in the A.I. project at Stanford University be continued
00080	and extended as a separate project with broadened aims in the field
00090	of speech understanding. This work gives considerable promise both of
00100	solving some of the immediate problems that beset speech
00110	understanding research and of providing a basis for future advances.
00120	
00130		It is further proposed that this work be more closely tied to
00140	the ARPA Speech Understanding Research effort than it has been in the
00150	past and that it have as its express aim the study and application to
00160	speech recognition of a machine learning process, that has proved
00170	highly successful in another application and that has already been
00180	tested out to a limited extent in speech recognition. The machine
00190	learning process offers both an automatic training scheme and the
00200	inherent ability of the system to adapt to various speakers and
00210	dialects. Speech recognition via machine learning represents a global
00220	approach to the speech recognition problem and can be incorporated
00230	into a wide class of limited vocabulary systems.
00240	
00250		Finally we would propose accepting responsibility for keeping
00260	other ARPA projects supplied with operating versions of the best
00270	current programs that we have developed. The availability of the high
00280	quality front end that the signature table approach provides would 
00290	enable designers of the various over-all systems
00300	to test the relative performance of the top-down portions of their
00310	systems without having to make allowances for the deficiencies
00320	of their currently available front ends. Indeed, if the signature table
00330	scheme can be made simple enough to compete on a time basis (and we
00340	believe that it can) then it may replace the other front end
00350	schemes that are currently in favor.
00360	
00370		Stanford University is well suited as the site for such work,
00380	having both the facilities for this work and a staff of people with
00390	experience and interest in machine learning, phonetic analysis, and
00400	digital signal processing. The staff at present consists of the
00410	proposed Principal Investigator Arthur l. Samuel, one post-doctoral
00420	staff member Ravindra Thosar,
00430	who had worked on speech  recognition and synthesis in India,
00440	a second member Dr. Neil Miller, who has had considerable signal-processing
00450	experience and a few graduate students. It is anticipated that a staff
00460	of not more than 3 full time members with the help of 3 or 4 graduate
00470	students could mount a meaningful program, which should be funded for a
00480	mimimum of two years to ensure continuity of effort.
00490	 We would expect to demonstrate the utility of the
00500	Signature Table approach within this time span and to provide a working
00510	system that could be used as the front end for any of the
00520	Speech Understanding Systems that are currently under
00530	development or are being planned.
00540	.
00550	
00560		Ultimately we would
00570	like to have a system capable of understanding speech from an
00580	unlimited domain of discourse and with an unknown speaker. It seems not
00590	unreasonable to expect the system to deal with this situation very
00600	much as people do when they adapt their understanding processes to
00610	the speakers idiosyncrasies during the conversation. The signature table
00620	method gives promise of contributing toward the solution of this
00630	problem as well as being a
00640	possible answer to some of the more immediate problems.
00650	
00660		The initial thrust of the proposed work would be toward the
00670	development of adaptive learning techniques, using the signature
00680	table method and some more recent varients and extentions of this
00690	basic procedure. We have already demonstrated the usefulness of this
00700	method for the initial assignment of significant features to the
00710	acoustic signals. One of the next steps will be to extend the method
00720	to include acoustic-phonetic probabilities in the decision process.
00730	
00740		Still another aspect to be studied would be the amount of
00750	preprocessing that should be done and the desired balance between
00760	bottom-up and top-down approaches. It is fairly obvious that
00770	decisions of this sort should ideally be made dynamicallly depending
00780	upon the familiarity of the system with the domain of
00790	discourse and with the characteristics of the speaker.
00800	Compromises will undoubtedly have to be made in any immediately
00810	realizable system but we should understand better than we now do the
00820	limitations on the system that such compromises impose.
00830	
00840		It may be well at this point to describe the general
00850	philosophy that has been followed in the work that is currently under
00860	way and the results that have been achieved to date. We have been
00870	studying elements of a speech recognition system that is not
00880	dependent upon the use of a limited vocabulary and that can recognize
00890	continuous speech by a number of different speakers.
00900	
00910		Such a system should be able to function successfully either
00920	without any previous training for the specific speaker in question or
00930	after a short training session in which the speaker would be asked to
00940	repeat certain phrases designed to train the system on those phonetic
00950	utterances that seemed to depart from the previously learned norm. In
00960	either case it is believed that some automatic or semi-automatic
00970	training system should be employed to acquire the data that is used
00980	for the identification of the phonetic information in the speech. We
00990	believe that this can best be done by employing a modification of the
01000	signature table scheme previously discribed. A brief review of this
01010	earlier form of signature table is given in Appendix 1.
01020	
01030		The over-all system is envisioned as one in which the more or
01040	less conventional method is used of separating the input speech into
01050	short time slices for which some sort of frequency analysis,
01060	homomorphic, LPC, or the like, is done. We then interpret this
01070	information in terms of significant features by means of a set of
01080	signature tables. At this point we define longer sections of the
01090	speech called EVENTS which are obtained by grouping together varying
01100	numbers of the original slices on the basis of their similarity. This
01110	then takes the place of other forms of initial segmentation. Having
01120	identified a series of EVENTS in this way we next use another set of
01130	signature tables to extract information from the sequence of events
01140	and combine it with a limited amount of syntactic and semantic
01150	information to define a sequence of phonemes.
01160	
01170		While it would be possible to extend this bottom up approach
01180	still further, it seems reasonable to break off at this point and
01190	revert to a top down approach from here on. The real difference in
01200	the overall system would then be that the top down analysis would
01210	deal with the outputs from the signature table section as its
01220	primitives rather than with the outputs from the initial measurements
01230	either in the time domain or in the frequency domain. In the case of
01240	inconsistencies the system could either refer to the second choices
01250	retained within the signature tables or if need be could always go
01260	clear back to the input parameters. The decision as to how far to
01270	carry the initial bottom up analysis must depend upon the relative
01280	cost of this analysis both in complexity and processing time and the
01290	certainty with which it can be performed as compared with the costs
01300	associated with the rest of the analysis and the certainty with which
01310	it can be performed, taking due notice of the costs in time of
01320	recovering from false starts.
01330	
01340		Signature tables can be used to perform four essential
01350	functions that are required in the automatic recognition of speech.
01360	These functions are: (1) the elimination of superfluous and
01370	redundant information from the acoustic input stream, (2) the
01380	transformation of the remaining information from one coordinate
01390	system to a more phonetically meaningful coordinate system, (3) the
01400	mixing of acoustically derived data with syntactic, semantic and
01410	linguistic information to obtain the desired recognition, and (4) the
01420	introduction of a learning mechanism.
01430	
01440		The following three advantages emerge from this method of
01450	training and evaluation.
01460		1) Essentially arbitrary inter-relationships between the
01470	input terms are taken in account by any one table. The only loss of
01480	accuracy is in the quantization.
01490		2) The training is a very simple process of accumulating
01500	counts. The training samples are introduced sequentially, and hence
01510	simultaneous storage of all the samples is not required.
01520		3) The process linearizes the storage requirements in the
01530	parameter space.
01540	
01550		The signature tables, as used in speech recognition, must be
01560	particularized to allow for the multi-category nature of the output.
01570	Several forms of tables have been investigated. Details of the current
01580	system are given in Appendix 2. For some early results  see 
01590	SUR Note 43 "Some Preliminary Experiments in Speech Recognition
01600	Using Signature Tables" by R.B.Thosar and A.L.Samuel.
01610	
01620	
01630		Work is currently under way on a major refinement of the
01640	signature table approach which adopts a somewhat more rigorous
01650	procedure. Preliminary results with this scheme indicate that a
01660	substantial improvement has been achieved. This effort is described in
01670	a recent report SUR Note 81 on "Estimation of Probability Density Using
01680	Signature Tables for Application to Pattern Recognition, by 
01690	R.B.Thosar.
01700	
01710	
01720		We are currently involved in work on a segmentation
01730	procedure which has already demonstrated its ability to compete with other
01740	proposed segmentation systems, even when used to process speech from 
01750	speakers whose utterances  were not used during the training
01760	sequence.
01770	
     

01415	
01420			Appendix 1
01430	
01440		The early form of a signature table
01450	
01460		For those not familiar with the use of signature tables as
01470	used by Samuel in programs which played the game of checkers, the
01480	concept is best illustrated (Fig.1) by an arrangement of tables used
01490	in the program. There are 27 input terms. Each term evaluates a
01500	specific aspect of a board situation and it is quantized into a
01510	limited but adequate range of values, 7, 5 and 3, in this case. The
01520	terms are divided into 9 sets with 3 terms each, forming the 9 first
01530	level tables. Outputs from the first level tables are quantized to 5
01540	levels and combined into 3 second level tables and, finally, into one

01550	third-level table whose output represents the figure of merit of the
01560	board in question.
01565	
01570		A signature table has an entry for every possible combination
01580	of the input vector. Thus there are 7*5*3 or 105 entries in each of
01590	the first level tables. Training consists of accumulating two counts
01600	for each entry during a training sequence. Count A is incremented
01610	when the current input vector represents a prefered move and count D
01620	is incremented when it is not the prefered move. The output from the
01630	table is computed as a correlation coeficient
01640	 			C=(A-D)/(A+D).
01645		The figure of merit for a board is simply the
01650	coefficient obtained as the output from the final table.
     

01670	
01680			Appendix 2
01690	
01700		Initial Form of Signature Table for Speech Recognition
01710	
01720		The signature tables, as used in speech recognition, must be
01730	particularized to allow for the multi-catagory nature of the output.
01740	Several forms of tables have been investigated. The initial form
01750	tested and used for the data presented in the attached paper uses
01760	tables consisting of two parts, a preamble and the table proper. The
01770	preamble contains: (1) space for saving a record of the current and
01780	recent output reports from the table, (2) identifying information as
01790	to the specific type of table, (3) a parameter that identifies the
01800	desired output from the table and that is used in the learning
01810	process, (4) a gating parameter specifying the input, that is to be
01820	used to gate the table, (5) the sign of the gate,
01825	 (6) the gating level to be used and (7)
01830	parameters that identify the sources of the normal inputs to the
01840	table.
01850	
01860		All inputs are limited in range and specify either the
01870	absolute level of some basic property or more usually the probability
01880	of some property being present. These inputs may be from the original
01890	acoustic input or they may be the outputs of other tables. If from
01900	other tables they may be for the current time step or for earlier
01910	time steps, (subject to practical limits as to the number of time
01920	steps that are saved).
01930	
01940		The output, or outputs, from each table are similarly limited
01950	in range and specify, in all cases, a probability that some
01960	particular significant feature, phonette, phoneme, word segment, word
01970	or phrase is present.
01980	
01990		We are limiting the range of inputs and outputs to values
02000	specified by 3 bits and the number of entries per table to 64
02010	although this choice of values is a matter to be determined by
02020	experiment. We are also providing for any of the following input
02030	combinations, (1) one input of 6 bits, (2) two inputs of 3 bits each,
02040	(3) three inputs of 2 bits each, and (4) six inputs of 1 bit each.
02050	The uses to which these differint forms are put will be described
02060	later.
02070	
02080		The body of each table contains entries corresponding to
02090	every possible combination of the allowed input parameters. Each
02100	entry in the table actually consists of several parts. There are
02110	fields assigned to accumulate counts of the occurrances of incidents
02120	in which the specifying input values coincided with the different
02130	desired outputs from the table as found during previous learning
02140	sessions and there are fields containing the summarized results of
02150	these learning sessions, which are used as outputs from the table.
02160	The outputs from the tables can then express to the allowed accuracy
02170	all possible functions of the input parameters.
02180	
02190	Operation in the Training Mode
02200	
02210		When operating in the training mode the program is supplied
02220	with a sequence of stored utterances with accompanying phonetic
02230	transcriptions. Each segment of the incoming speech signal is
02240	analysed (Fourier transforms or inverse filter equivalent) to obtain
02250	the necessary input parmeters for the lowest level tables in the
02260	signature table hierarchy. At the same time reference is made to a
02270	table of phonetic "hints" which prescribe the desired outputs from
02280	each table which correspond to all possible phonemic inputs. The
02290	signature tables are then processed.
02300	
02310		The processing of each table is done in two steps, one
02320	process at each entry to the table and the second only periodically.
02330	The first process consists of locating a single entry line within the
02340	table as specified by the inputs to the table and adding a 1 to the
02350	appropriate field to indicate the presence of the property specified
02360	by hint table as corresponding to the phoneme specified in the
02370	phonemic transcription. At this time a report is also made as to the
02380	table's output as determined from the averaged results of previous
02390	learning so that a running record may be kept of the performance of
02400	the system. At periodic intervals all tables are updated to
02410	incorporate recent learning results. To make this process easily
02420	understandable, let us restrict our attention to a table used to
02430	identify a single significant feature say Voicing. The hint table
02440	will identify whether or not the phoneme currently being processed is
02450	to be considered voiced. If it is voiced, a 1 is added to the "yes"
02460	field of the entry line located by the normal inputs to the table. If
02470	it is not voiced, a 1 is added to the "no" field. At updating time
02480	the output that this entry will subsequently report is determined by
02490	dividing the accumulated sum in the "yes" field by the sum of the
02500	numbers in the "yes" and the "no" fields, and reporting this quantity
02510	as a number in the range from 0 to 7. Actually the process is a bit
02520	more complicated than this and it varies with the exact type of table
02530	under consideration, as reported in detail in appendix B. Outputs
02540	from the signature tables are not probabilities, in the strict sense,
02550	but are the statistically-arrived-at odds based on the actual
02560	learning sequence.
02570	
02580		The preamble of the table has space for storing twelve past
02590	outputs. An input to a table can be delayed to that extent. This table
02600	relates outcomes of previous events with the present hint-the
02610	learning input. A certain amount of context dependent learning is thus
02620	possible with the limitation that the specified delays are constant.
02630	
02640		The interconnected hierarchy of tables form a network which
02650	runs increamentally, in steps synchronous with time window over which
02660	the input signal is analised. The present window width is set at 12.8
02670	ms.(256 points at 20 K samples/sec.) with overlap of 6.4 ms. Inputs
02680	to this network are the parameters abstracted from the frequency
02690	analyses of the signal, and the specified hint. The outputs of the
02700	network could be either the probability attached to every phonetic
02710	symbol or the output of a table associated with a feature such as
02720	voiced, vowel etc. The point to be made is that the output generated
02730	for a segment is essentially independent of its contiguous
02740	segments. The dependency achieved by using delayes in the inputs is
02750	invisible to the outputs. The outputs thus report the best estimate on
02760	what the current acoustic input is with no relation to the past
02770	outputs. Relating the successive outputs along the time dimension is
02780	realised by counters.
02790	
02800	The Use of COUNTERS
02810	
02820		The transition from initial segment space to event space is
02830	made posible by means of COUNTERS which are summed and reiniated
02840	whenever their inputs cross specified threshold values, being
02850	triggered on when the input exceeds the threshold and off when it
02860	falls below. Momentary spikes are eliminated by specifying time
02870	hysteresis, the number of consecutive segments for which the input
02880	must be above the threshold. The output of a counter provides
02890	information about starting time, duration and average input for the
02900	period it was active.
02910	
02920		Since a counter can reference a table at any level in the
02930	hierarchy of tables, it can reflect any desired degree of information
02940	reduction. For example, a counter may be set up to show a section of
02950	speech to be a vowel, a front vowel or the vowel /I/. The counters can
02960	be looked upon to represent a mapping of parameter-time space into a
02970	feature-time space, or at a higher level symbol-time space. It may be
02980	useful to carry along the feature information as a back up in those
02990	situations where the symbolic information is not acceptable to
03000	syntactic or semantic interpretation.
03010	
03020		In the same manner as the tables, the counters run completely
03030	independent of each other. In a recognition run the counters may
03040	overlap in arbitrary fashion, may leave out gaps where no counter has
03050	been triggered or may not line up nicely. A properly segmented output,
03060	where the consecutive sections are in time sequence and are neatly
03070	labled, is essential for processing it further. This is achieved by
03080	registering the instants when the counters are triggered or
03090	terminated to form time segments called events.
03100	
03110		An event is the period between successive activation or
03120	termination of any counter. An event shorter than a specified time is
03130	merely ignored. A record of event durations and upto three active
03140	counters, ordered according to their probability, is maintained.
03150	
03160		An event resulting from the processing described so far,
03170	represents a phonette - one of the basic speech categories defined as
03180	hints in the learning process. It is only an estimate of closeness to
03190	a speech category , based on past learning. Also each category has a
03200	more-or-less stationary spectral characterisation. Thus a category may
03210	have a phonemic equivalent as in the case of vowels , it may be
03220	common to phoneme class as for the voiced or unvoiced stop gaps or it
03230	may be subphonemic as a T-burst or a K-burst. The choices are based on
03240	acoustic expediency, i.e. optimisation of the learning rather than
03250	any linguistic considerations. However a higher level interpretive
03260	programs may best operate on inputs resembling phonemic
03270	trancription. The contiguous events may be coalesced into phoneme like
03280	units using diadic or triadic probabilities and acoustic-phonetic
03290	rules particular to the system. For example, a period of silence
03300	followed by a type of burst or a short friction may be combined to
03310	form the corresponding stop. A short friction or a burst following a
03320	nasal or a lateral may be called a stop even if the silence period is
03330	short or absent. Clearly these rules must be specific to the system,
03340	based on the confidence with which durations and phonette categories
03350	are recognised.
03360